kourouklidesfandomcom-20200213-history
Deep Learning
This page contains resources about Deep Learning and Representation Learning . Subfields and Concepts * Deep Generative Models ** Deep Directed Networks (directed graphical models) *** Sigmoid Belief Net *** Differentiable Generator Net *** Variational Autoencoder (VAE) *** Generative Adversarial Network (GAN) *** Generative Moment Matching Network *** Convolutional Generative Network *** Auto-Regressive Network / Fully-visible Bayes Network (FVBN) *** Deep Latent Gaussian Model (DLGM) *** Deep AutoRegressive Network (DARN) ** Deep Boltzmann Machines (undirected graphical models) ** Deep Belief Networks (mixed graphs) * Deep Neural Networks (i.e. more than two hidden layers) ** Deep Multi-Layer Perceptron (i.e. Stacked RBMs) ** Deep Autoencoders (i.e. two symmetrical DBN) *** DARN ** Deep Neural Decision Forests ** Convolutional Deep Belief Network (i.e. Stacked CRBMs) * Sparse Coding / Dictionary Learning ** Sparse Autoencoders ** Stacked Denoising Autoencoders * Bayesian Deep Learning ** Bayesian Neural Networks Online Courses Video Lectures *Introduction to Deep Learning - Coursera *Neural Networks for Machine Learning by Geoffrey Hinton - Coursera *Deep Learning Specialization by Andrew Ng - Coursera *Deep Learning by Google - Udacity *Neural networks class by Hugo Larochelle (Youtube ) *Deep Learning and Neural Networks by Kevin Duh *Computer Perception with Deep Learning by Yann LeCun (Part 1 , Part 2 ) *Computational Neuroscience and Learning by Eugenio Culurciello (Youtube) *A tutorial on Deep Learning by Geoffrey Hinton - VideoLectures.Net *Deep Learning with TensorFlow: Applications of Deep Neural Networks to Machine Learning Tasks by Jon Krohn Lecture Notes *Practical Deep Learning for coders by Jeremy Howard *Deep Learning by Yann LeCun *Unsupervised Feature Learning and Deep Learning (UFLDL) by Andrew Ng *Deep Learning by Bhiksha Raj *Representation Learning by Yoshua Bengio *Convolutional Neural Networks for Visual Recognition by Fei-Fei Li & Andrej Karpathy *Deep Learning by Sargur Srihari *DataLab Cup 5: Deep Reinforcement Learning *Deep Learning by Shan-Hung Wu Books and Book Chapters * Santana, E. (2018). Eder Santana's Deep Learning with Python. Packt Publishing. * Shukla, N. (2018). Machine learning with TensorFlow. Manning. * Zaccone, G., Karim, Md. R., & Menshawy, A. (2017). Deep Learning with TensorFlow. Packt Publishing. * McClure, N. (2017). TensorFlow Machine Learning Cookbook. Packt Publishing. * Gulli, A., & Pal, S. (2017). Deep Learning with Keras. Packt Publishing. * Chollet, F. (2017). Deep Learning with Python. Manning Publications. * Gulli, A., & Kapoor, A. (2017). TensorFlow 1.x Deep Learning Cookbook. Packt Publishing. * Bengio, Y., Goodfellow, I. J., & Courville, A. (2016). Deep Learning. MIT Press. (link) * Gibson, A., & Patterson J. (2016). Deep Learning: A Practitioner's Approach. ''O'Reilly Media. * Nielsen, M. A. (2015). ''Neural Networks and Deep Learning. Determination Press. * Theodoridis, S. (2015). "Chapter 18: Neural Networks and Deep Learning". Machine Learning: A Bayesian and Optimization Perspective. Academic Press. * Odense, S. (2015). Universal approximation theory of neural networks. MSc Diss. University of Victoria. * Du, K. L., & Swamy, M. N. (2014). Neural networks and statistical learning. Springer Science & Business Media. * Deng, L., & Yu, D. (2014). Deep Learning. Foundations and Trends in Signal Processing, 7'', 3-4. * Bengio, Y., & Courville, A. (2013). ''Deep Learning of Representations. Springer. * Barber, D. (2012). "Chapter 26: Distributed Computation". Bayesian Reasoning and Machine Learning. Cambridge University Press. * Neal, R. M. (2012). Bayesian learning for neural networks. Springer Science & Business Media. * Orr, G. B., & Muller, K. R. (2012). Neural Networks: Tricks of the Trade. Springer. * Murphy, K. P. (2012). "Chapter 28: Deep Learning". Machine Learning: A Probabilistic Perspective. MIT Press. * Alpaydin, E. (2010). "Chapter 11: Multilayer Perceptrons". Introduction to Machine Learning. MIT Press. * Haykin, S. S., Haykin, S. S., Haykin, S. S., & Haykin, S. S. (2009). Neural Networks and Learning Machines. 3rd Ed. Pearson. * Bengio, Y. (2009). Learning Deep Architectures for AI. Foundations and Trends® in Machine Learning, 2(1), 1-127. Now Publishers. * LeCun, Y., Chopra, S., Hadsell, R., Ranzato, M., & Huang, F. (2006). "A Tutorial on Energy-Based Learning". Predicting Structured Data. MIT Press. * Bishop, C. M. (2006). "Chapter 5: Neural Networks". Pattern Recognition and Machine Learning. Springer. * MacKay, D. J. (2003). "Chapter 38: Introduction to Neural Networks" Information Theory, Inference and Learning Algorithms. Cambridge University Press. * Mandic, D. P., & Chambers, J. (2001). Recurrent neural networks for prediction: learning algorithms, architectures and stability. John Wiley & Sons. * Rojas, R. (1996). Neural networks: a systematic introduction. Springer Science & Business Media. (link) * Bishop, C. M. (1995). Neural networks for pattern recognition. Oxford University Press. * Hassoun, M. H. (1995). Fundamentals of artificial neural networks. MIT press. Scholarly Articles See Reading List and Recommended Readings for the complete list. * Schmidhuber, J. (2015). Deep learning in neural networks: An overview. Neural networks,Volume 61, 85-117. * Paul, A., & Venkatasubramanian, S. (2014). Why does Deep Learning work?-A perspective from Group Theory. arXiv preprint arXiv:1412.6621. * Shwartz-Ziv, R., & Tishby, N. (2017). Opening the Black Box of Deep Neural Networks via Information. arXiv preprint arXiv:1703.00810. * Chakraborty, S., Tomsett, R., Raghavendra, R., Harborne, D., Alzantot, M., Cerutti, F., ... & Kelley, T. D. (2017). Interpretability of Deep Learning Models: A Survey of Results. Tutorials *Information Theory of Deep Learning by Prof Naftali Tishby *UFLDL Tutorial *A Deep Learning Tutorial: From Perceptrons to Deep Networks - with Java examples *Neural Networks, Manifolds, and Topology - advanced *Deep Learning Tutorials by Olver Dürr - Lasagne and TensorFlow *Efficient Deep Learning with Humans in the Loop by Zachary Chase Lipton *Deep Generative Models by Durk Kingma (2014) *Learning Invariant Feature Hierarchies (2013) and its Panel Discussion *Deep Learning Tutorial (ICML 2013) *Deep Learning for Computer Vision (NIPS 2013) *Deep Learning for NLP (NAACL 2013) *Deep Support Vector Machines (ROKS 2013) *Deep Learning of Representaions (SSTiC 2013) *Deep Learning for Machine Vision (BMVC 2013) *Deep Learning for Computer Vision (NIPS 2013) (Video ) *Deep Learning Methods for Vision (CVPR 2012) *Deep Learning for NLP (ACL 2012) *Representation Learning (ICML 2012) *Classification with Deep Invariant Scattering Networks (NIPS 2012) *A tutorial on deep and unsupervised feature learning for activity recognition (2011) *Tutorial on Deep Learning and Applications (NIPS 2010) *A tutorial on Deep Learning (2009) *Tutorial on Learning Deep Architectures (ICML 2009) *Learning Deep Hierarchies of Representations (2009) *Deep Learning with Multiplicative Interactions (NIPS 2009) *Learning Feature Hierarchies (MLSS 2009) *Deep Belief Networks (MLSS 2009) Software See Software Links for the complete list. *TensorFlow - Python *Theano - Python (discontinued; no longer supported or developed) *Keras - Python library for TensorFlow and Theano *Lasagne - Lightweight library to build and train neural networks in Theano *Caffe - C/C++, Python, MATLAB, command line *Torch - Lua *PyTorch - Python and C++ *fastai - Python *pydlt - PyTorch based Deep Learning Toolbox *CNTK - The Microsoft Cognitive Toolkit *cuDNN - CUDA GPU library supporting TensorFlow, Theano, Torch, Caffe, Keras, CNTK and others *TFLean - Deep Learning Python library featuring a higher-level API for TensorFlow *Blocks - A Theano framework for training neural networks *Pylearn2 - Python *deeplearning4j - Java *MXNet - MATLAB, Python, C++, R, Julia, Scala, Go, Javascript and more *Chainer - Python *CudaCnn - MATLAB *hebel - Python *ConvNetJS - Deep Learning models (mainly Neural Networks) entirely in your browser *OpenNN - C++ *visual-rbm *MatCovNet - MATLAB *Learning Deep Boltzmann Machines - MATLAB *Estimating Partition Functions of RBM's - MATLAB *Deep Belief Networks - MATLAB *DeepLearnToolbox - MATLAB/Octave *Netlab neural network software - MATLAB *Neural Network Toolbox - MATLAB *PyBrain - Python *DyNet - Python and C++ *handson-ml - Python *DLL - C++ *DeepRosetta - Python *torchgeometry - Python See also *Deep Learning in the news (blog) *Probabilistic Graphical Models *Sparse Coding Other Resources * Deep Learning with Python by Jason Brownlee - practical book * Deep Learning Reading List * DeepLearning.Net - Tutorials and a general point of reference * Toronto Deep Learning Demos - source code *Deep learning from the bottom up - Metacademy *Neural Networks and Deep Learning - free online book *Awesome-Deep-Vision (Github) - A curated list of Deep Learning resources for Computer Vision *Awesome-Deep-Learning (Github) - A curated list of resources *Awesome-Deep-Learning-papers (Github) *Awesome-TensoFflow (Github) *Deep Learning Libraries by Language *Deep Learning (Building Intelligent Probabilistic Systems) - Blog by Harvard University *Benchmark of Deep Learning Representations for Visual Recognition *Deep Learning Playlist - Youtube collection of video lectures and tutorials *Deep Learning on Google+ - online community *A Short History of and Introduction to Deep Learning - Presentation by John Kaufhold *An Introduction to Deep Learning: From Perceptrons to Deep Networks - tutorial with Java examples *Graduate Summer School: Deep Learning, Feature Learning by IPAM, UCLA *What Does a Neural Network Actually Do? - Neural Networks and Deep Learning *Ersatz - Deep Neural Networks in the cloud *Bibliography in Deep Learning - collection of papers categorized according to type of application *Deep Learning Papers Reading Roadmap *What My Deep Model Doesn't Know - Blog post on Bayesian Neural Networks *New Theory Cracks Open the Black Box of Deep Learning *Why does Deep Learning work? by Charles H Martin - blog post *Why Deep Learning Works II: The Renormalization Group by Charles H Martin - blog post *Practical_DL - Github *deep-learning-book - Github *Deep Learning: Theory & Practice - blog post Category:Machine Learning Category:Computational Neuroscience